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Computers have become a solution for many traditional problems from the kitchen to the school. It is not, 
therefore, uncommon to use them to solve existing problems. It is also quite frequent that different 
educational needs can lead to similar solutions. For instance, the Australian immigration services use a 
number of exams to allow the access of qualified professionals into the country. Likewise, the TOEFL 
exam is commonly used as a requirement for foreign students to access to American universities. 
Similarly many European universities that receive a large number of international students every year 
have developed their own language test to admit or place foreign students. 

The Universidad Politecnica de Valencia is probably among the top three universities in Spain by the 
incoming number of international students. Only in the last three years the Gandia College (UPV) has 
tripled the total number of these students. Thus, creating and developing especially tailored courses for 
them is quite a complex issue. One of the reasons is the content teachers’ limited knowledge of foreign 
languages but also, probably the most important, the difficulty of having reliable information of the 
international students proficiency level upon arrival. Many universities use their own language services to 
assess the national and international students but these tests are usually free of charge for the students and 
are costly in human and economic resources, and consequently many universities tend to use very limited 
resources to diagnose the incoming students. In addressing this matter, GILFE, a research group in 
languages and technology at the Universidad Politecnica de Valencia, has been studying this problem for 
three years and two years ago started a research project to develop a computer based test to do this job 
(Garcia Laborda, 2004; Garcia Laborda & Bejarano, 2005). This computer tool was thought to be Internet 
based and would be available for low stake testing and is currently in trial. 
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1. General Introduction 

Computers have become a solution for many traditional problems from the kitchen to the school. It is 
not, therefore, uncommon to use them to solve existing problems. It is also quite frequent that different 
educational needs can lead to similar solutions. For instance, the Australian immigration services use a 
number of exams to allow the access of qualified professionals into the country. Likewise, the TOEFL 
exam is commonly used as a requirement for foreign students to access to American universities. Simi- 
larly many European universities that receive a large number of international students every year have 
developed their own language test to admit or place foreign students. 

The Universidad Politecnica de Valencia is probably among the top three universities in Spain by the 
incoming number of international students. Only in the last three years the Gandia College (UPV) has 
tripled the total number of these students. Thus, creating and developing especially tailored courses for 
them is quite a complex issue. One of the reasons is the content teachers’ limited knowledge of foreign 
languages but also, probably the most important, the difficulty of having reliable information of the in- 
ternational students proficiency level upon arrival. Many universities use their own language services to 
assess the national and international students but these tests are usually free of charge for the students 
and are costly in human and economic resources, and consequently many universities tend to use very 
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limited resources to diagnose the incoming students. In addressing this matter, GILFE, a research group 
in languages and technology at the Universidad Politecnica de Valencia, has been studying this problem 
for three years and two years ago started a research project to develop a computer based test to do this 
job[l][2]). This computer tool was thought to be Internet based and would be available for low stake 
testing and is currently in trial. 

However, although the needs were somehow different, as soon as the first version of the new iBT 
TOEFL appeared (see <www.ets.org/toefi> for further information) the similarities were self evident. 
This paper will try to present the tool and also show the differences between the two platforms. It is not 
in the scope of this paper to address the differences of content and testing procedures between them but 
to point out the concept differences between both testing platforms. There is little question that further 
research is necessary in relation to the development of new language platforms and, in this sense, this 
paper is only a tentative approach to such an attractive issue. 

2. Integrating skills 

Given the basic and initial needs to develop PLEVALEX (Platform of Evaluation of Foreign Lan- 
guages), it was self evident the need to develop a tool that could integrate all the skills that students 
would need in their social and academic lives. It seems unnecessary to mention that all four skills should 
be integrated in the computer platform. These skills should be integrated in realistic environments. It 
should also be relevant to the students’ realities and needed to include certain types of oral and written 
exercises that could replace the traditional pen-and-paper and pair-interview tasks that were so common 
until the beginning of the 2004-2005 academic year. The incorporation of academic mini-lectures in the 
exam made also think the developers the possibility of incorporating note taking as a need. Up to here, 
the coincidences with the new iBT TOEFL changes are also clear. The inclusion of these tasks responds 
to the fact that the traditional tests and oral exercises had a very limited relationship with the university 
routines that include attending classes, writing well planned papers, take exams of out-of-classroom 
readings but also in-class lectures. The exams distributed by PLEVALEX seem to ensure a fair but also 
useful diagnosis. 

What is the internal organization? 

As opposed to the TOEFL, the PLEVALEX platform allows the student a great flexibility. This paper 
will not describe each section in depth that can be found in other studies (Garcia Laborda, 2006) but it is 
also important to mention that the examinee can choose in which order to proceed when interacting with 
the index: 1) grammar / reading (HIELE), 2) writing (ESCRITOR), and 3) speaking (HIEO). HIELE 
uses multiple choice questions mostly, ESCRITOR includes as many writing essays as required (usually 
1 or 2), and HIEO has short social or academic questions and longer descriptions. It is mostly in this last 
section where students will be freer to use notes taking at the time of playing the videos in the speaking 
section. TOEFL has four sections but will be done in order: reading, listening, speaking and writing*. 
PLEVALEX designers, researchers and administrators strongly believe as those in TOEFL that obtaining 
a threshold grade in the tests distributed with these two platforms can be considered reliable as the tests 
will match most academic demands required in higher education. It is self evident in both platforms that 
communication has overtaken grammatical accuracy and language use, and that criticism of the validity 
of language testing as a predictor of language proficiency can be clearly override. This position is 
strongly supported because the idea of PLEVALEX’s designers is to achieve a flexible and adaptative 
test in a near future and being able to equate each student against the Common European Framework 
(CEF) instead of having the currently developed norm based exam at B2/3 proficiency level. From the 
overall diagnosis based on this proficiency level, it is expected to obtain valuable information to be used 



' Specimens of the different types of tasks that are mentioned in this paper can be found in the website mentioned above likewise in 
[3] the students will also find a complete selection of interfaces used in the exam. 
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by the different institutions across Europe and also provide feedback for international student mobility 
preparation (washback). 

Exam sections 

The new testing platform can deliver a competitive test that can measure the international students arriv- 
ing to that university. To make a short description, the tool is composed by three modules: written com- 
position, multiple choice and a speaking skills sections. 



Section^ 


Multiple Choice 


Written Composition 


Speaking section 


Tasks ^ 


Choose one (correet) 
answer 


Short answers 
Prompted or figure 
aided compositions 


General pair directed 
questions 

Lecture and question 
answer 

Diagram / picture de- 
scription 



Fig.l ; Exam sections 



As mentioned before, the platform integrates all four skills and follows the same general structure 




Fig.2: Interface design for exams. Please note the different parts of the composition section 

This structure is very simple and tries to avoid any possible interference due to any stimuli not directly 
related to the exam and the minimum requirements to represent the prompts, question of space for re- 
sponse. The main difference in the design stage is that while PLEVALEX uses limited elements specifi- 
cally developed for this platform, all the elements included in TOEFL were specifically design for their 
test. Of course, this has to be with the limited budget that the Spanish platform had totalling 24,000 Euro. 

This schemata was used for the writing sections mostly by both platforms and, in relation to 
PLEVALEX, it was also used for the speaking section without the space for writing and being substi- 
tuted by an ON/OFF speaking icon. 

In relation to the “language use" / grammar, both platforms used single question interfaces. 
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Fig. 3: Multiple choice or reading section 



Other tan that and as it has been made clear by the interfaces shown in this short paper are clear so, how 
does PLEVALEX outcome TOEFL? Is it worth to develop platforms like PLEVALEX? 

Possibilities for PLEVALEX and further developments 

It is self evident that the limitation of author tools like TOEFL makes them extremely expensive for low- 
stakes exams. Universities could produce tools like this to evaluate large quantities of students when the 
results can have a limited scope for the examinees. For example, the Universidad Politecnica de Valencia 
will be using and testing this tool in the Foreign Language Institute (that means, as opposed to TOEFL, 
at least in 6 languages) in the first semester of 2006. Usability tests with Spanish and international stu- 
dents totalling 35 reports and questionnaires have proved that the platfomi is easy to use and compre- 
hend. Further studies should consider how to use this platfomi for high stakes exams like the National 
Entrance Examination, defining a theory of language test interface and platform design, students’ differ- 
ences due to the use of keyboard (versus pen and paper), individual strategies, and many more. 

Plevalex and TOEFL face to face 

When preparing the test and platform, the researchers intended to compare this exam mainly to the new 
version of the TOEFL to which the exams distributed through HIEO (Herramienta informatica de 
Evaluacion Oral) and HIELE (Plataforma de Evaluacion de Lenguas Extranjeras) resemble. The follow- 
ing diagrams in the poster summarise the differences between both platforms. 

Conclusion 

The PLEVALEX platform has an enormous potential as it can integrate all four skills plus grammar in a 
controlled and easy-to-distribute test. It also helps to show semi-communicative skills and situations. Its 
format also benefits the using institutions as it allows a better use of the testing budget by using it for 
raters the more human tasks like grading speaking and writing. Although this complex project still re- 
mains unfinished the short and long run perspectives are very positive. The prospective studies in exami- 
nee satisfaction and comfort will certainly bring new improvements to this software. 
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Appendix 1: The poster 



PLEVALEX 

Starting Date 2003 (open info) 

Budget 24,000€ 

Subjects International / Erasmus 

Students 

Goals Improve entrance exams, lower 

pnce per placement (teachers, resources), fair 
evaluation through use of four skills 




Design 



Starting Date 



Budget: 

Subjects: 
going to US, 

Goals: 



TOEFL 

2001 (secret into, first 
publications by sections) 
Unknown, maybe millions 

International Students 
AUS universities 



Improve reliability in TOEFL, 
increase prices wbile lowering human resources, fair 
evaluation through use of four skills, integrated total 
exam 




Skm ■ < •» *• dMM »r MM 

Impact Erasmus. Spanish students 
Authenticity DrfTerent registers 
Reliability: Reliable tasks 

Validity: Similar measures to current 
practices but more tunes 



.. Computer based, mos 
students perform better 



Process 



Test 
features 

I 

Plevalex and the 
new TOEFL: 



i TOEFL iBT I 
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